Input devices 



120 



Output devices 




Code 
module A 
(CS) 



Code 
module B 
(CC) 



Code 
module C 
(CS/CC) 



Code 
module D 
(CS/CC) 
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Fig. 3 




300 



232 



Initialize CS and/or CC 
for theme set for user 



402 



At CS, obtain data signals from input devices 



At CS, process data signals to produce 
attributes representing user's context, 
including adding required data fields, such as 
attribute name, timestamp, units, etc. 



At CS, provide attribute to CM for storage 



At CC, request from CM and receive attributes 



404 



406 



408 



410 



At CC, process attributes to determine appropriate output 



Provide output signal to user, other entity or other process 



412 



414 



Fig. 4 



400 



Launch at start-up 



502 



Periodically receive GPS data stream 



504 



Parse and store GPS data 



508 

"Receive" 
request for latitude,^"-^ n 
.longitude or altitude, 
.attribute?^ 



506 



Provide requested attribute value(s), 
with Timestamp, uncertainty, units, etc. 



Fig. 5 A 
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Name 



Latitude (value) 



Uncertainty 



Timestamp 



Units 



Format version/flags 



521 
522 
524 
526 
528 
530 



Fig. SB 



Register for latitude, longitude and altitude attributes 



602 



604 




606 



Use default attribute mediator 



Reevaluate all latitude attributes 



608 



Select lowest uncertainty attribute mediator 



Determine CS providing resulting attributes 



610 



•612 



Request latitude, longitude and altitude attributes from CS/CM 



616 




Receive input defining boundary 



'618 



Provide event parameters to CM regarding boundary transversal 



622 

Receive 
'response from CM to^>N 
^established event? 



Notify user that boundary has been crossed 
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Fig* 6 
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User. 

Desired_privacy_level 

Interruptibility 

Speed 

Direction 

Acceleration 

Availability. 

Cognitive_availability 

Tactile_availability 

Manual_availability 

Visual_availability 

Qral_availability 

Auditory_availability 
Proximity. <Item or place name> 
Mood. 

Happiness 

Sadness 

Anger 

Frustration 

Confusion 
Activity. 

Driving 

Eating 

Running 

Sleeping 

Talking 

Typing 

Walking 
Location. 

Place_name 

Latitude 

Longitude 

Altitude 

Room 

Floor 

Building 

Address 

Street 

City 

County 

State 

Country 

Postal_Code 
Destination, (same as User.Location.) 
Physiology. 

Pulse 

Body_temperature 

Blood_pressure 

Respiration 
Person. <name or ID>. (same as User.) 
Platform. 
UL 

Oral_input_device_availability 

Manual_input_device_availability 

Tactile_output_device_availability 

Visual_output_device_availability 

Auditory_output_device_availability 



Fig. 8 



Platform, (continued) 
CPU. 

Load 

Speed 
Memory. 

Total_capacity 

Used 
Storage. 

Total_capacity 

Used 
Connectivity. 

Connection_status 

Connection_speed 

Connection_type/device 

Connection_activity 
Power. 

Power_source 

Power__level 
Environment. 
People. 

Nearest 

Number_present 
Number_close 
Local. 

Time 
Date 

Temperature 
Pressure 
Wind_speed 
Wind_direction 
Absolutejiumidity 
High_forecast_temperature 
Low_forecast__temperature 
People_present 
Ambient_noise_level 
Ambient_light_level 
Days.<previous or future>. 
High_temperature 
Low_temperature 
Precipitation_type 
Precipitation_amount 
Place. <place name> (same as Environment. Local) 
Application. 
Mail 

Available 

New_messages_waiting 
Messages_waiting_to_be_sent 
Phone. 

Available 

In_use 

On/off 

Notification jnechanism 
Call_incoming 
CallerJD 
Sound_recorder. 
Available 
Recording 



User Setting Computer Fig* 9 

Mental Context Power 

Meaning Configuration 

Cognition User Input Systems 

Divided User Attention Hand/Haptic 

Task Switching Keyboard/Keystrokes 

Background Awareness Voice/ Audio 

Solitude Eye Tracking 

Privacy Cursors 

Desired Privacy Axis 

Perceived Privacy Resolution 

Social Context Selection 

Affect Invocation 

Physical Situation Accelerators 

Body Output Systems 

Biometrics Visual 

Posture Resolution 

Motion Audio 

Physical Encumberment Public/Private 

Senses Haptic 

Eyes External Resources 

Ears I/O devices 

Tactile Connectivity 
Hands Data 

Nose Quantity/State 

Tongue Urgency/Importance 

Workload demands/effects Use of Prominence 

Interaction with computer devices Modality 

Interaction with people Sensitivity/Purpose 

Physical Health Privacy Issues 

Environment Use of Association 

Time/Space Use of Safety 

Objects Source/Ownership 

Persons Types 

Audience/Privacy Availability User generated 

Scope of Disclosure Other computers or people 

Hardware affinity for privacy Sensor 

Privacy Indicator for User PC State 

Privacy Indicator for Public Use of Association 
Watching Indicator 
Being Observed Indicator 
Ambient Interference 
Visual 
Audio 
Tactile 
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Example Current Theme Selection 



Select A New Current Theme 



1150 



/--Starting The Day 

' —Driving Home 
-At Home 

Work-Related Group ' 
-Driving To Work 
-Arriving At Work 
-At Work 




> 1155 



Entertainment-Related Group 



OK 



Cancel 



1157 



Figure 11M 



Starting The Day 

1150 ^ At Home 

.Eating Breakfast 



Example Current Theme Set Modification 
Current Theme Set 



Add Theme 



1157 



Remove Theme 



Figure UN 



Example Current Theme Layout Selection 

Select A New Current Theme Layout 

Starting The Day theme 
f Starting The Day - default layout 

1 1 60 < Starting The Day - vacation 

L Starting The Day - responsible for dropping off kids 



OK 



1157 



Cancel 
ZZ — 



Figure 110 





1302 


1304 




c 
) 




Example Theme Data Structure 

1.™ 


1305^\^ 


Name 




1310-"X> 


Description 


uuem is 1 o wrgamze rsotn jjnvmg into oc Into About Work Arrival 




Security 




1320^^ 


Permission 


/\LLCbb. oen t ramuy -i- rnenas + oource, Moaiiication. belt; Copy: None; . , 


1325^N^> 


Privacy 




1330^\> 


Priority 


Medium 


1335^N^ 


Group-ID 


23 


1340^\_, 


Time-Active 


Mon-Fri, 06:00-11:00 


1345'-N^ 


Source 


BCD Motor Company 




Theme-Content 


"BCD Motor logo.gif ' 


1355-^ 


New-Attributes 


DTW-BCD.driving-direction: (source = "BCD-DTW CS tf ); 


1360^s^ 


Theme-Matching 


user.activity.driving = true (required); envkonmentlocal.thrie = 08:00-09:00 (optional); 
DTW-BCD. driving-direction = NOT "South" (required); . . . 


1365^^ 


Theme-Logic 


IF <user.activity.driving> AND user.location.latitude > recent user, location.latitude 

THEN DTW-BCD.driving-direction = "North"; 
IF DTW-BCD.driving-direction = "East" THEN AlertfYou're Lost Again! !"); 
SET user.locationfAccessl = Family & Co-workers; . . . 


1370^. 


Attribute-Set 


<link v >; ♦ ♦ * <link v > 




• 
• 
• 




\ 



r Attribute 1 



Name 


user.activity.driving 


Value 


True 


Uncertainty 


High 


Timestamp 


07:32 03/24/XX 


Units 


boolean 


Access 


Family 





Attribute N 



Name 


DTW-BCD.driving-direction 


Value 


West 


Uncertainty 


None 


Timestamp 


08:13 03/24/XX 


Units 


North, North by Northeast, . . . 


Access 


Self 





Fig. 13 
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Fig. 15 



1500 

Theme usage 
routine 



1505 



Receive indications of currently available themes 
and their associated theme information 



1510 



Retrieve current context information 



1515 



Determine themes that match current context 



1530 



Present matching or all 


4 Yes 


themes to user 




> 


r 1535 




Receive indication of 




selected theme 





1520 



User selection? 



No 



1525 



Select highest priority 
matching theme 



1540 



Generate appropriate 
response to selected theme 



1545 



Detect changes to context or user indication 



1555 



Retrieve context changes 



v 1550 



No, 



User indication? 



1565 



Present current context 
information to user 





Yes 




r 1560 



Yes, 



Change context 
information? 



.No 




1570 



Receive indication of 
context changes 



1595 



END 



Yes 1575 



Exit? 



1580 




No 


Provide indicated 


<— 




functionality to user 







Fig. 16 



1615 



Select default theme layout 
if multiple are associated 



1620 



Retrieve content specified 
by theme layout 



1540^ 

r Response generator 
subroutine 





r 1605 


Receive indication of selected theme 
and of associated theme information 




r 1610 



Yes, 



Associated theme 
layout(s)? 



V 1625 

Perform data logic specified 
by theme layout 



1630 



Retrieve user preference 
information 



± 1635 

Present content in manner specified by presentation logic 
of theme layout and in a manner consistent with 
preference information 



1640 



Other response 
specified by theme 
layout? 



v 1645 



Provide other response as 
appropriate 



it 1695 



Return 



1700 

Theme creator/ 
modifier routine 





r 1705 




p> 




Receive indication of theme 
to be created or modified 


1715 




' 1710 



Fig. 17 



1720 




1725 



Display theme to user 



1730 



Receive indication from user to modify 
specified properties of theme or 
information associated with theme 



1740 



Present theme layout to user 
in visual editor 



1735 



,Yes, 



Modify theme 
layout? 





No 




t 1750 



1745 



Modify property? \^es ^ 



Modify theme layout as 
indicated by user 



No 



1755 



Modify theme property as 
indicated by user 



1760 



Modify theme attribute, 
theme CS, or theme CC as 
indicated by user 



1765 



Yes. 



More changes to 
theme? 



No 



1770 



Store theme and 
theme-related information 



Yes. 



v 1775 



More themes? 



1795 



.No 



END 



1800^ 

Theme distributor 
routine 



1805 



Fig. 18 



Receive indications of 
available themes 



j 1810 

Retrieve indications of 
appropriate users for the 
available themes 



1815 



Receive request for theme(s) 
or indication of accessible 
user 



1835 



Retrieve information about 
user from database or user's 
computing device 



v 1820 



No, 



Request? 



^Yes 



1825 



Determine if user has 
provided appropriate access 
information 



1840 



1830 



Determine themes that are 
appropriate for user 



Yes y 



Appropriate 
access? 



1845 




Determine if any needed 
payment information has 
been provided 



1850 



Payment 
provided? 



No 



Yes 



1855 



Gather theme-related information 
associated with requested or determined 
themes, including the themes 



1860 



Send gathered information 
to requesting or indicated 
user 



1865 



Yes, 



Continue? 





No 




r 1895 



END 



1900 

Theme receiver 
routine 



1905 



Receive user request for 
theme or indication of sent 
theme 



Fig. 19 



1910 



No. 



> 


(, 

' 1940 


Receive indication of sent 
theme 




r 1945 



Request? 



.Yes 



1915 



Attempt to determine theme 
server that can provide 
requested theme 



1920 



No, 



Payment 
information 
needed? 



Yes 



1950 



Send payment information 



1955 



Payment accepted? 



No 



Yes 



1960 



Receive theme and any 
associated theme-related 
information 



No, 



n 1922 



Indicate failure 
to user 



1965 



Store received information, 
and load for use if 
immediately useful 



>r 1970 



Yes. 



Continue? 



Theme server 
found? 



Yes 



1925 



Receive appropriate access 
information for theme and 
theme server, either from 
user or from storage 



1930 



Receive indication of 
payment mechanism if 
needed, either from user or 
from storage 



1935 



Send request for theme to 
theme server, including any 
access information and 
payment information 





No 




r 1995 



END 



2000 



Automated theme 
customizer routine 



Receive indications of available themes 




r 2010 



Repeatedly monitor user actions, 
including interactions with themes, 
modifications to themes, and explicit 
changes to the current theme or the 
current context 





r 2015 


Analyze monitored 
patterns ( 


actions to determine 
>f actions 



No 



2020 



Any patterns above relevance 
threshold? 





Yes 

r 2025 


Retrieve user preference information 




r 2030 


Determine modifications to some or all 
themes that are consistent with patterns 
above the relevance threshold and with 
user preferences 




r 2035 



Do automated modification? 



No 



1 


Yes 

r 2040 


Modify themes as determined 




r 2045 


Notify user of modifications 




2065 



Yes 



Continue? 





No 


1 


r 2095 



END 



Fig. 20 



2050 



Present suggested modifications to the 
user 



2055 



Receive indications of accepted 
modifications 



2060 



Modify themes as indicated 



o 

*5b 
o 
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Fig. 24 




- Verbal - 

— GPS — 
—Mouse — 
■ Keyboard- 
— Time — 



-Message Loop* 



Aging & 
Compression 



Actions 



Modify 
Windows 



Manage PIM 



Initiate calls, 
pages, faxes, 
email, web 
searches 



Display Maps 



more . 



Filters & 
Builds Datastore 



Preference 
Pattern 



A 

yr 



Journal 



Pattern 
Recognition 



Inference 
Engine 



Rules 



Time 



Place 



Command 



File 



Learning 
Preferences 



User Defined 



.2500 



Context-Based Automated 
Learning Routine 



Fig. 25 



2505 



Receive context information 



2510 



Add context information to explicit 
context model, and provide responses 
as indicated by explicit context rules 




r 2515 


Provide context information from 
explicit context model to implicit 
context model with inference engines 




r 2520 


Detect patterns in explicit context 
information, and suggest inferred rule 
with appropriate response 


> 


r 2525 


Verify appropriateness of suggested 
rule 




r 2530 



No 



Appropriate? 



Yes 



2535 



Store suggested rule, and provide 
suggested response if appropriate 



Yes 



v 2540 



Continue? 



No 



2595 



END 



Self-Customizing ContexT 
Awareness Routine 



2600 



Fig. 26 



2605 



Receive current context information 



± 2610 

Receive indication of modeled context attribute 
information from explicit context model that 
reflects received context 



± 2615 

Receive indication of user selection of new 
modeled context attribute information or of 
indication of appropriate modeled context 
attribute information 



2620 



Add all received information (and changes in 
current context from previous context) to 
appropriate context awareness customizing 
implicit model and inference engine 




r 2625 


Detect pattern in indicated user selection or 
indication, and generate corresponding inferred 

rule to change modeled context attribute 
information in appropriate customized manner 




r 2630 


Verify appropriateness of generated rule 




r 2635 



Appropriate? 



Yes 



2640 



Store generated rule for future use 



Yes 



< 



2645 



Continue? 



2695 



END 



Predict Appropriate 
Content Routine 



2700 



2705 



Receive current context information 




t 2710 


Receive indication of user selection of 
information or user indication of 
appropriate information 




r , 2715 



Add current context information 
(including changes from previous 
context) and indicated user selection 
or indication to appropriate content 
implicit model and inference engine 





r 2720 


Detect pattern in indicated user 
selections of information and generate 
corresponding inferred rule to provide 
appropriate information 


> 


t 2725 


Verify appropriateness of generated 
rule 




>800 



Self-Optimizing UI Routine) 



> 


r 2805 




Receive current context information 




t 2810 


Receive indication of user changes to 
UI functionality or user indication of 
appropriate UI functionality 




t 2815 


Add received information (including 
changes from previous context) to 
appropriate UI optimizing implicit 
model and inference engine 




r 2820 


Detect pattern in user changes or 
indications, and generate 
corresponding inferred rule to provide 
appropriate UI functionality 




r 2825 


Verify appropriateness of generated 
rule 




r 2830 



No 



Appropriate? 



Yes 



2835 



Store generated rule for future use 



Yes 



yr 2840 



Continue? 



No 



2900 



Task Simplification 
Routine 



>> 


r 2905 




Receive current context information 




r 2910 


Receive indication of user task 
performed having multiple steps, or of 
user indication that task is appropriate 




r 2915 


Add received information (including 
changes from previous context) to 

appropriate task simplification 
implicit model and inference engine 




r 2920 


Detect pattern in task, and generate 
corresponding inferred rule to provide 
some or all of appropriate task 




r 2925 


Verify appropriateness of generated 
rule 




r 2930 



No 



Appropriate? 



Yes 



2935 



Store generated rule for future use 



Yes 



v 2940 



Continue? 



No 



3000 



Mentoring Routine^ 



3005 



Fig. 30 



Receive current context information 



3010 



Yes 



Explicit mentoring 
rules? 



No 



3015 



Retrieve explicit mentoring rules 



3035 



3020 



Determine if match to current 
context, and if so prompt user as 
indicated 



3025 



After prompt, 
receive "Why?" 
question 



Indication from user? 



Yes 



3030 



Generate and provide 
response based on 
explicit rules 



No 



3040 



Receive indication from 
user of user action that 
should have been 
performed based on 
current context other 
than any prompted 
actions 



yr 3045 



Add received information (including changes from 
previous context) to appropriate mentoring implicit 
model and inference engine 



3050 



Detect pattern in indicated user action and current 
context information, and generate inferred rule to 
provide indicated action in context for pattern 



3055 



Verify appropriateness of generated rule 



v 3060 



Appropriate? 



No 



Yes 



3065 



Modify explicit rules (or store inferred rule if no 
explicit rules) to reflect inferred rule 



Yes 



yr 3070 



Continue? 



No 



3095 



END J 



